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Abstract 



Results pertaining to methodological aspects of an aptitude- 
treatment Interaction study are presented. One treatment 
group In the study was given Imagery Instructions^ thia other 
had no Imagery Instructions. Each treatment group consisted 
of seven flfth*grade classes. Among the aptitude variables 
there were two versions of a palred*assoclates learning 
task. Hlthln-rclass analyses and analyses where class effects 
were allowed to have Influence were conducted. Iii the latter 
analyses there were several, significant aptitude*treatitient 
Interactions with subscofes derived from the palred-assocl- 
ateis tasks f buz this was. not the case in the v/lthln-class 
analyses. The Interactions found are Interpreted as being 
consequences of class effects with respect to the errors of 
measurement . 
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2. The totals are referred to as Pictures T and Words T. On each 
occasion half the number of the other aptitude variables were 
also administered* On the second occasion the experiment proper 
was the first activity of the session. On both occasion the PA 
test was administered as the 'last test* 

Since the data collection took place on two occasions-inhere was 
an attrition of the treatment groups* According to the class 
lists there were 169 and 173 pupils in the Hi and I groups re- 
spectively. Since only subjects with a complete set of data were 
included/ there were 141 and 130 subjects respectively in the 
treatments that were available for analysis. 

Procedures in the analysis 

Analyses were conducted allowing for curvilinear regressions of 
the dependent variables on each aptitude variable within treat* 
ments* This was effected through the addition of a quadratic 
term* One set of analyses (labeled Pooled) were made in which 
within-class and between-class effects were not separated* In 
these analyses the raw scores on the dependent variables were 
used/ and for the aptitude variables deviation scores round the 
total. mean were used* The rationale for using deviation scores 
rather than raw scores was to avoid multicol linearity with the 
quadratic variable* 

In their reanalysis of the Anderson study Cronbach and Webb 
(1975) used for the within-class analysis deviation scores round 
class means for the dependent variables as well as for the apti^ 
tude variables* Classes were pooled, thereby allowing each class 
to influence the analysis according to its size* Here the same 
procedure has been followed in a set of analyses called Within* 

No estimates of any further effects have been made* Even though 
between-class regression could have easily been determined, 
comparisons between the pooled and the Within analyses suffice 
for the present purposes* 

Significance testing presents great problems in the analysis of 
within-class and between-class effects* For one thing a large 



number of classes is needed if a significant heterogeneity of 
between'^-'class within-treatment regressions is ever to be .found* 
For another thing the construction of proper error terms presents 
problems. The recognition that the class is a unit, both in sampl- 
ing and in treatment, of course reveals as illegitimate the usual 
procedure in what is here called the Pooled analyses, of acting 
as if the individual was the sampling unit* Despite this, however, 
the number of pupils in the classes was used for determining the 
degrees of freedom for the error mean square in the Pooled anal* 
yses. The main reason for this was that no alternative was avail- 
able. The same degrees of freedom have been used for the Within 
analyses in spite of the fact that each class evidently consumes 
one degree of freedom. However, correcting for this woul<3 here 
have only little bearing on the pattern of results. 

The statistical tests and the estimation of coefficients of regres 
sion were made under a "general linear hypothesis" model, with 
treatment coded as a dummy variable and aptitude-treatment inter- 
action effects represented with cross-^product terms in the regres- 
sion equations. The computations were performed with program 
BMDIOV (Dixon, 1973) . 



RESULTS 
* 

Results from the Pooled analyses are presented in Table 1. 
Several significant interaction effects are found. The verbal 
test Opposites shows a significant difference in slope between 
treatments with respect to the Simple criterion; rnd with respect 
to the same criterion there are for Metal folding significant 
differences between the treatments with respect to both the linear 
and the quadratic terms. For the PA variables there are in several 
cases differences between the treatments for the quadratic terms. 
But it is quite embarrasing to find that the parallel versions of 
the same test give very different patterns of results. For the 
Words 1 variable there is for example with respect to the Complex 
criterion a highly significant interaction; but for the Words 2 
variable there is not only a lack of significant interaction but 
there are also for both the linear and the quadratic terms weak 
tendencies in the opposite direction to what was found with Words 
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Table 1 Coefficients of regression and F-ratios for ap,titude-tr§at;^ 
ment interactions in the Pooled analyses. '2 



Simple 
NI I 



Complex 
NI I 



Description 
NI I F 



Opposite s 



Metal 
folding 



X .154 .260 

x'-.OlO -.010 

F for overall 
interaction 

X .065 .139 

x' .003 -.003 

F for overall 
interaction 



Words 1 



.186 



.286 



.038 -.029 

F for overall 
interaction 



Pictures 1 



Words 2 



X .166 .206 

x' .034 -.011 

F for overall 
interaction 

X .258 .059 

x^-.002 .027 

F for overall 
interaction 



Pictures 2 



X .227 
x^ .014 



.096 
.026 



Words T 



F for overall 
interaction 

X .167 .120 

x' .005 -.003 

F for overall 
interaction 



Pictures T 



X .136 .088 

x^ .007 -.002 

F for overall 
interaction 



5.46"" .152 .132 .44 

.00 .003 .000 .41 

2.82 .52 

4.34^* .046 .063 .52 

5.91^* .001 -.004 2.70 

5.90^* 1.80 

.45 .152 .302 2.46 

2.17 *.049 -.055 12.93* 

1.09 6.b2* 

.07 .064 .230 2.98 

.68 .013 .006 .03 

.55 1.93 

2.38 .234 .159 .80 

.60 -.002 .002 .02 

1.19 .46 

1.21 .201 .101 1.66 

.09 .050 -.009 4.67^ 

.80 2.59 

.34 .138' .149 .05 

,40 .010 -.008 5.69^ 

.73 3.46^ 

.42 .107 .099 .03 

.28 .020 -.009 6.55^ 

.28 3.40^ 



,312 
,007 



,148 
,000 



,109 
,047 



,018 
,025 



,250 
,007 



,343 
,035 



,136 
,010 



166 



,220 2,51 
,006 .02 

1.27' 

,131 .13 
,006 1.21 

.63 

.377 1.98 
.033 2.00, 

1,32 

.366 3.65 
.051 .15 

2.59 



,237 
,015 



,265 



.202 



.01 
.03 

.02 

.28 



-.025 4.65' 

2.33 

.227 .81 
-.010 1.65 

.87 



.16 



,030 -.029 7.50 



4.39' 



Critical values: F gg (1,265)=3.88 F gg {2, 265) =3 .03 



Table 2 Coefficients of regression and F-ratios for aptttude-treat- 
ment interactions in the Within analyses. 



Nl 



Simple 
I 



Conplifx 
NI I 



[description 
NI I P 



Opposites 



Metal 
folding 



Words 1 



Pictures 1 



Words 2 



Words T 



X .116 .248 

x^-.005 -.006 

F for overall 
interaction 

X .055 .143 

x^ .008 -.002 

F for overall 
interaction 

X .276 .286 

x^-.004 -.015 

F for overall 
interaction 

X .222 .203 

x^ .032 -.019 

F for overall 
interaction 



X .205 
x^-.OOl 



.14 7 
.020 



F for overall 
interaction 



Pictures 2 



X .186 
x^-.013 



.157 
.015 



F for overall 
interaction 



X .166 
x^-.OOl 



.136 
.002 



F for overall 
interaction 



Pictures T 



X .130 
x^-.007 



.120 
.001 



F for overall 
interaction 



7.87' 
.05 

4.02* 

6.23* 
4.59* 

5.69^ 

.00 
.08 

.04 

.02 
.93 

.49 

.23 
.30 

.19 

.06 
.48 

.37 

.17 
.07 

.09 

.02 
.21 

.10 



,122 
,004 



,133 
,001 



,039 
,001 



,067 
,004 



,192 
,019 



,268 
,047 



,117 
,019 



,227 
,002 



,190 
,011 



,189 
,001 



,171 
,026 



,147 
,001 



,131 
,004 



,147 
,007 



,100 
,010 



,123 
,001 



.12 
1.37 

.69 

1.40 
1.93 

1.75 

.67 
5.89* 

3.05* 

1.56 
.27 

1.30 

.00 
.23 

.14 

.10 
.85 

.42 

.11 
1.92 

1.00 

.27 
.70 

.63 



.266 
,006 



,115 
,000 



.098 
,025 



,305 
,049 



,118 
,001 



,151 
,009 



,232 
,011 



,177 .283 
,008 .020 



.35 
.30 

.42 



,158 1.00 
,009 2.37 

1.77 



,150 .383 1.69 
,011 -.016 .28 



.86. 

,381 2.89 
,059 .29 

2.18 



.53 
.35 

.75 



.376 .25 
-.014 1.60 

1.28 

.233 1.58 
-.005 .12 

.81 

.257 1.53 
-.014 1.78 

2.21 



Critical values: T gg (1, 265) ==3.88 , F gg (2,265)=3.03 
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Results from the Within analyses are presented in Table 2. The 
interactions found for both Opposites and Metal folding reappear 
essentially unaltered in these analyses. But almost all the inter 
actions with the PA variables fail to reappear in the Within 
analyses. The only exception is formed by the Words 1 variable 
with respect to the Complex criterion, but it can be noted that 
this interaction is considerably weaker in the Within analysis. 

DISCUSSION AND CONCIUSIGNS 

Here will only be discussed methodological aspects of the pattern 
of results; the reader interested in a full presentation of the 
results and interpretations in substantive terms for the verbal 
and spatial aptitude variables is referred to Gustafsson (1976 
b) . • 

The fact that for the PA variables there were no interactions 
in the Within analyses, and the fact that there were great 
differences between the patterns of results of the two forms f 
indicate that the results obtained in the Pooled analyses are 
nothing else than artefacts arising from the circumstance that 
the administration of the PA test as well as the experiment 
proper took place with intact classes. 

It is reasonable to assume that a learning task is highly sen- 
sitive to the instructions given f to the mental alertness of 
the subjects, to events happening during the administration 
of the task and so on. When the task is given class-wise this 
implies that many of these factors will influence all pupils 
more or less in the same way. The effect of this, of course, 
is that there will be a large intraclass correlation for the 
errors of measurement. But when this occurs for both aptitude 
variables and dependent variables in an ATI study where classes 
are nested within treatments this may imply correlated errors 
for aptitude variables and dependent variables. Since these 
correlated errors may be of different kinds within the treat- 
ments this may in turn have as a consequence what is judged to 
be significant aptitude-treatment interactions. 



In the Within analyses, in. contrast, each subjects scores on 
the variables are computed as deviations from the class means; 
any positive or negative fcvtuitious effect on the scores that 
is common to the class will consequently not affect the anal- 
ysis . 

In this case the class effects with respect to the errors of 
measurement resulted in quite complex aptitude-treatment 
interactions which pertained to the quadratic component of 
the regression. Prom this it cannot be assumed, however, that 
the linear regressions on the aptitude variables remain free 
from such effects* Whenever there is a small number of classes 
that are nested within treatments, class effects at the ad* 
ministration~of^the aptitude variables and experimental tasks 
can have as a consequence interactions of ?ny kind. 

* 

The preseat analysis thus strongly indicates the need for 
keeping track of classy-mediated effects also with respect to 
the" aptitude variable s, Aptitude variables (and classes) of 
course are sensitive in differing degrees to such effects, but 

whenever the presence of class-medxated errors of measurement 
are suspected, within-class analyses should be conducted. 

Obviously attempts should also be made to avoid the problem 
altogether. One possibil^ty is, of course, to administer 
''sensitive" tests individually* This, however, may not be 
possible due to the large number of subjects needed for apti- 
tude-treatment interaction studies* If for this, or any other 
reason, strictly individual testing is not possible, (small) 
groups of subjects having different treatments should be tested 
together * 



r 
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